Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

most lint stuff seems unrelated to PR. pytest part as said still WIP.
Input welcomed

pytest fixed but a new external error on tests/test_pkg_imports.py::test_conda_reqs
https://github.com/microsoft/msticpy/actions/runs/14564764340/job/40852428655?pr=839#step:10:129

pytest fixed but a new external error on tests/test_pkg_imports.py::test_conda_reqs https://github.com/microsoft/msticpy/actions/runs/14564764340/job/40852428655?pr=839#step:10:129

@juju4
Fix should be very simple - just add this requirement to the /conda/conda-pip-reqs.txt

This test is just designed to keep the requirements/setup packages aligned so it's saying that there is something in requirements/setup that's not reflected in the /conda requirements files.
Since this isn't going to be downloadable from Conda or Conda-forge, it should go in the conda-reqs-pip.txt (which is packages that cannot be installed by conda install and have to be installed using pip)

seems conda package can't be a commit. hope to have a pypi release next week-end

Current pylint errors not related to PR
https://github.com/microsoft/msticpy/actions/runs/14824892002/job/41616735671?pr=839#step:9:13

ImportError: cannot import name 'UTC' from 'datetime' is a difference between python 3.9 and 3.10. is replacing code by following ok or other preference depending on 3.9 support plan?

from datetime import datetime, timedelta, timezone
UTC = timezone.utc

done that for now.

3.8 fails with /home/runner/work/_temp/960b5084-0bdd-4156-9364-2d9478b37221.sh: line 1: jupyter: command not found
https://github.com/microsoft/msticpy/actions/runs/14825003555/job/41616982508?pr=839#step:10:18

Only python 3.8 ci is failing and seems unrelated to PR
https://github.com/microsoft/msticpy/actions/runs/15232868365/job/42842868008?pr=839#step:7:278

Collecting cryptography>=43.0.1 (from -r requirements-all.txt (line 19))
  Downloading cryptography-45.0.1-cp37-abi3-manylinux_2_34_x86_64.whl.metadata (5.7 kB)
  Using cached cryptography-44.0.3-cp37-abi3-manylinux_2_34_x86_64.whl.metadata (5.7 kB)
ERROR: Exception:
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 106, in _run_wrapper
    status = _inner_run()
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 97, in _inner_run
    return self.run(options, args)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 67, in wrapper
    return func(self, options, args)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 386, in run
    requirement_set = resolver.resolve(
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/pip/_vendor/resolvelib/resolvers.py", line 457, in resolve
    raise ResolutionTooDeep(max_rounds)
pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 200000
Error: Process completed with exit code 2.

@ianhelle How to get this PR move forward?
python_openobserve requires python>=3.10. not sure if a way to exclude tests of lower version or plan to up minimal msticpy python.

lint failing on external conditions for installing dependencies

× Dependency resolution exceeded maximum depth
╰─> Pip cannot resolve the current dependencies as the dependency graph is too complex for pip to solve efficiently.

Hey Julien,
I'm thinking of dropping Python <=10 anyway - also want to add Py 3.13 and maybe Py 3.14.
In the short term you can do with a combo of the following:

Add OpenObserve as an Extra in setup.py (follow the examples in here)
In there add a constraint for the python version like this one.

ipython >= 7.23.1; python_version >= "3.8"

Also add that to the requirements_all.txt and the conda files.

You will also need to guard the import of open observe - I've added comments to that file with an example.
Finally, you can skip the tests using pytest.mark.skipif - I've also added an example in a comment in the test module.

still blocked on installing dependencies.
added the python constraint in requirements-all.
I believe openobserve part is already set as an extra. missing link for examples.
Thanks

I fixed the extra - it needs to have a python_version in the extra declaration as well as the requirements.
I will check out the PR and have a look to see if I can see what's breaking.

I added some changes to the requirements/setup.py for python-openobserve.
Also pytest skips if the package is not installed.
This will continue to fail until openobserve is fixed - I opened a PR to do this here JustinGuese/python-openobserve#64.
When this PR is completed it should still work.
We might want to merge this into the msticpy 3.0 release (which drops support for Python 3.8 and 3.9) - depending on when the openobserve PR is completed.

PR merged and 0.4.2 released. let's see how tests are doing

lint 3.12 and build 3.11 failing for same reason. build 3.9 ok (not in scope/ignored)

 × Dependency resolution exceeded maximum depth
╰─> Pip cannot resolve the current dependencies as the dependency graph is too complex for pip to solve efficiently.

@juju4
I've re-merged this with the Msticpy 3.0 main. That should (I think) get rid of a lot of the package incompatibility issues.
Let's see what the build says.

Things are a little better but this is our fundamental problem:

Will try a different candidate, due to conflict:
    azure-kusto-data 5.0.0 depends on requests==2.32.0
    azure-core 1.37.0 depends on requests>=2.21.0
    msal 1.34.0 depends on requests<3 and >=2.0.0
    folium 0.20.0 depends on requests
    geoip2 5.2.0 depends on requests<3.0.0 and >=2.24.0
    msrest 0.7.1 depends on requests~=2.16
    adal 1.2.7 depends on requests<3 and >=2.0.0
    panel 1.7.5 depends on requests
    python-openobserve 0.4.2 depends on requests<2.33.0 and >=2.32.4

I think it's the version conflict between the requests versions between azure-kusto-data (seems a bit crazy that they are locked to single version!!). and python-openobserve. Although the msrest requirement is also a long way away from the python-openobserve.

azure-kusto-data 5.0.0 depends on requests==2.32.0
and
azure-kusto-data 5.0.0 depends on requests==2.32.0

Is there a good reason for it being locked so tight? Usually you see very specific versions for apps but not so much for libraries that are meant to be installable/compatible with many versions.
Msitcpy stopped using requests a long time ago since it's old and not well supported. An option would be to switch openobserve to use httpx. Another would be investigating loosening the version range - seems like the openobserve code doesn't do anything weird - just a few get and post requests. So, not sure why there is such a narrow dependency.
I could say the same about azure-kusto-data but I likely have less traction with that team.

requests latest is 2.32.5/Aug 2025 and 2.32.4/Jun 2025 minimum is because of CVE-2024-47081. 2.32.0 goes back to May 2024.
httpx switch is on the radar (JustinGuese/python-openobserve#20). I just pushed initial code but not confident ready for msticpy 3.0 unless few weeks away.
On the other side azure-kusto-data (https://github.com/Azure/azure-kusto-python) latest is 6.0.1 based on aiohttp or 5.0.5/Jul 2025 with either requests 2.32.3 and aiohttp as extra. So that's the path I would recommend and added in latest commit. That seems to do the trick.

Thanks a lot @ianhelle